Method of filtering a pixel of an image

ABSTRACT

In one embodiment, a pixel in a block of the image is filtered according to a filtering methodology. The filtering methodology includes determining a filtering strength value. This determining step determines a first filter strength value if the block has non-zero transform coefficient information and determines a second filter strength value if a motion vector difference between a motion vector of the first block and a motion vector of an adjacent second block is greater than or equal to one. The first filter strength value is greater than the second filter strength value.

DOMESTIC PRIORITY INFORMATION

This is a divisional of U.S. application Ser. No. 10/170,999 filed Jun.14, 2002 now U.S. Pat. No. 7,272,186; the contents of which are herebyincorporated by reference in their entirety.

FOREIGN PRIORITY INFORMATION

The present invention claims priority under 35 U.S.C. 119 on KoreanApplication No. 2001-33953 filed Jun. 15, 2001; the contents of whichare hereby incorporated by reference in their entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a moving picture compression technique,and more particularly, to a loop filtering method in a video coder.

2. Description of the Background Art

Generally, it has been proved that the “H.26L” system developed as anext generation moving picture compression technique is superior to the“H.263” and “MPEG4” systems in performance.

The “H.26L” system as the next generation moving picture compressionsystem differs from the conventional H.263 and MPEG4 in using 4×4 blockbased transformation & encoding, carrying out motion estimation &compensation of a transformation block size, and using a single variablelength coder (VLC).

Superior to the conventional motion picture standards in aspect ofperformance, the H.26L system has excessive calculation of encoder aswell as uses 4×4 block based transformation so that a blocking dominatesthan the conventional standards.

In order to remove the blocking, filters include a loop filtering systeminside the encoder and decoder, a post filter processed in a next stageof the decoder, and a system adding the loop filtering system to thepost filter.

When the blocking is removed by the loop filtering system in general, adifference between an input video and a motion video is affected so thatan encoding rate increases in a specific case. Yet, as the loopfiltering system is installed inside a video coder, complexity of theencoder increases.

As mentioned in the above explanation, the H.26L system according to therelated art requires excessive quantity of calculation despite thesuperiority in performance to the moving picture standards as well ashaving the blocking more dominant than that of the moving picturestandards due to the 4×4 block based transformation system.

Moreover, if the blocking is removed by the loop filtering systemaccording to the related art, the difference between the input video andthe motion video is affected so as to increase the encoding rate in aspecific case. Yet, the loop filtering system built inside the videocoder increases the complexity of the encoder.

SUMMARY OF THE INVENTION

The present invention provides methodologies for filtering a pixel of animage.

In one embodiment, a pixel in a block of the image is filtered accordingto a filtering methodology. The filtering methodology includesdetermining a filtering strength value. This determining step determinesa first filter strength value if the block has non-zero transformcoefficient information and determines a second filter strength value ifa motion vector difference between a motion vector of the first blockand a motion vector of an adjacent second block is greater than or equalto one. The first filter strength value is greater than the secondfilter strength value.

In one embodiment, the filtering methodology filters the pixel based onthe filtering strength value.

In another embodiment, the filtering methodology filters the pixel basedon the filtering strength value, the pixel to be filtered and at leastone pixel adjacent to the pixel to be filtered.

In an further embodiment, the filtering methodology filters the pixelbased on the filtering strength value, the pixel to be filtered, a firstpixel adjacent to the pixel to be filtered, a second pixel adjacent tothe pixel to be filtered, and a quantization parameter of the block.

In an embodiment, the non-zero transform coefficient information is atleast one non-zero transform coefficient.

It is to be understood that both the foregoing general description andthe following detailed description of the present invention areexemplary and explanatory and are intended to provide furtherexplanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this application, illustrate embodiment(s) of the invention andtogether with the description serve to explain the principle of theinvention. In the drawings:

FIG. 1 illustrates a flowchart of a loop filtering process according toan embodiment of the present invention;

FIGS. 2A and 2B illustrate block diagrams indicating block locations inhorizontal and vertical directions according to an embodiment of thepresent invention, respectively;

FIG. 3 illustrates a diagram of pixel locations of adjacent blocksaccording to an embodiment of the present invention; and

FIG. 4 illustrates a table of LQP values for determining DQP(QP) valuesaccording to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE EXAMPLE EMBODIMENTS

Reference will now be made in detail to the example embodiments of thepresent invention, which are illustrated in the accompanying drawings.

Hereinafter, a loop filtering method in a video coder according to anexample embodiment of the present invention carries out a real-timeprocessing of removing a blocking and a ringing phenomenon in accordancewith a pixel location, a coded block pattern, a quantization step size,and a motion vector size in a digital video appliance using the H.26Lmoving picture compression technique, thereby enabling to improve avideo quality of a compression image demanding a low bit rate orhigh-speed processing.

First, explained in detail is a real-time processing technique ofremoving “blocking” and “ringing” by predicting an original video forcorrelation between pixels in accordance with a quantization variableQP, a coded block pattern, and a motion vector size available for adecoder (not shown in the drawing).

When an original video f is transmitted as a compressed form, a videorestructured in a decoder can be expressed by the following Formula 1-1.g=f+n,  [Formula 1-1]where g, f, and n are a compression video rearranged in a scanningorder, an original video, and a column vector of quantization error,respectively.

A relation between respective pixels can be expressed by the followingFormula 1-2 in order to process the Formula 1-1 by pixel unit.g(i,j)=f(i,j)+n(i,j),  [Formula 1-2]where i and j indicate a location of a pixel in vertical and horizontaldirections.

An additional function can be defined as the following Formula 1-3 whenreliability of original pixels and 1-dimensional smoothing function foreach pixel are used together with a regularization. Firstly, ahorizontal direction is considered.M{f(i,j)}=M _(p) {f(i,j)}+M _(N) {f(i,j)},  [Formula 1-3]where M_(p) and M_(N) are functions representing reliability smoothingdegrees of left and right directions for a pixel f(i,j), respectively.

And, such functions are defined as the following Formula 1-4, where theM_(P) and M_(N) are the functions considering f(i,j) & f(i,j−1) andf(i,j) & f(i,j+1), respectively.M _(P) {f(i,j)}=α_(P) {f(i,j)−f(i,j−1)}²+(1−α_(P)){g(i,j)−f(i,j)}²M _(N) {f(i,j)}=α_(N){f(i,j)−f(i,j−1)}²+(1−α_(N)){g(i,j)−f(i,j)}²  [Formula 1-4]

Each of the first terms of right sides in Formula 1-4 indicatesirregularity with an adjacent pixel, i.e. a difference from an adjacentpixel, for the pixel f(i,j), and each of the second terms indicates areliability for the original pixel f(i,j). Moreover, α_(P) and α_(N)mean regularization parameters representing ratios between irregular andreliability terms defined in the functions, respectively. And, α_(P) andα_(N) are parameters adjusting a relative coherence between the twoterms in Formula 1-4.

With the above manner, the additional function is defined for each pixelof the moving picture, and the function in a vertical direction isdefined by varying the parameter i instead of the parameter j in Formula1-4.

Thereafter, if a differential is applied to the additional functions inFormula 1-4 defined for each pixel of the moving picture in order tofind a restored video from which the blocking and ringing are removed,the following Formulas 1-5 and 1-6 are attained.

$\begin{matrix}{\frac{{\partial M}\left\{ {f\left( {i,j} \right)} \right\}}{\partial{f\left( {i,j} \right)}} = {{\frac{{\partial M_{P}}\left\{ {f\left( {i.j} \right)} \right\}}{\partial{f\left( {i,j} \right)}} + \frac{{\partial M_{N}}\left\{ {f\left( {i,j} \right)} \right\}}{\partial{f\left( {i,j} \right)}}} = 0}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}5} \right\rbrack\end{matrix}$

From Formula 1-5, the following Formula 1-6 is found.

$\begin{matrix}{{\frac{{\partial M_{P}}\left\{ {f\left( {i,j} \right)} \right\}}{\partial{f\left( {i,j} \right)}} = {{2{\partial_{P}\left\{ {{f\left( {i,j} \right)} - {f\left( {i,{j - 1}} \right)}} \right\}}} - {2\left( {1 - \alpha_{P}} \right)\left\{ {{g\left( {i,j} \right)} - {f\left( {i,j} \right)}} \right\}}}}{\frac{{\partial M_{N}}\left\{ {f\left( {i,j} \right)} \right\}}{\partial{f\left( {i,j} \right)}} = {{2{\partial_{N}\left\{ {{f\left( {i,j} \right)} - {f\left( {i,{j - 1}} \right)}} \right\}}} - {2\left( {1 - \alpha_{N}} \right)\left\{ {{g\left( {i,j} \right)} - {f\left( {i,j} \right)}} \right\}}}}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}6} \right\rbrack\end{matrix}$

From Formulas 1-5 and 1-6, the pixel to be restored in a horizontaldirection can be found by the following Formula 1-7.

$\begin{matrix}{{f\left( {i,j} \right)} = \frac{{\left( {2 - \alpha_{P} - \alpha_{N}} \right){g\left( {i,j} \right)}} + {\alpha_{P}{f\left( {i,{j - 1}} \right)}} + {\alpha_{N}{f\left( {i,{j + 1}} \right)}}}{2}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}7} \right\rbrack\end{matrix}$

As the regularization parameters in Formula 1-7 are between “0” and “1”,Formula 1-7 can be defined by the following Formula 1-8.

$\begin{matrix}{{f\left( {i,j} \right)} = \frac{{\left( {2 - \alpha_{P} - \alpha_{N}} \right){g\left( {i,j} \right)}} + {\alpha_{P}{g\left( {i,{j - 1}} \right)}} + {\alpha_{N}{g\left( {i,{j + 1}} \right)}}}{2}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}8} \right\rbrack\end{matrix}$

Looking into Formula 1-8, a blocking-removed video of the compressionvideo is determined by the two, left and right pixel values and theregularization parameters by taking the i_(th) and j_(th) videos asreferences.

As the two, left and right pixel values can be used for encoding anddecoding, two regularization parameter values are set up for attainingthe restored video. For this, “set theoretic” is applied to therestoration by pixel unit so as to represent regularization coefficientsby the following Formula 1-9, where “set theoretic” is a theory forcontrolling the parameters α_(P) and α_(N) in Formula 1-4.

$\begin{matrix}{\begin{matrix}{\frac{\alpha_{P}}{1 - \alpha_{P}} = \frac{\left\{ {{f\left( {i,j} \right)} - {g\left( {i,j} \right)}} \right\}^{2}}{\left\{ {{f\left( {i,j} \right)} - {f\left( {i,{j - 1}} \right)}} \right\}^{2}}} \\{{\approx \frac{K_{P}{QP}^{2}}{\left\{ {{g\left( {i,j} \right)} - {g\left( {i,{j - 1}} \right)}} \right\}^{2}}},}\end{matrix}\begin{matrix}{\frac{\alpha_{N}}{1 - \alpha_{N}} = \frac{\left\{ {{f\left( {i,j} \right)} - {g\left( {i,j} \right)}} \right\}^{2}}{\left\{ {{f\left( {i,j} \right)} - {f\left( {i,{j + 1}} \right)}} \right\}^{2}}} \\{\approx \frac{K_{N}{QP}^{2}}{\left\{ {{g\left( {i,j} \right)} - {g\left( {i,{j + 1}} \right)}} \right\}^{2}}}\end{matrix}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}9} \right\rbrack\end{matrix}$

K_(P) and K_(N) in Formula 1-9 are determined statistically as adaptiveparameters depending on a location of a pixel since the irregularitiesbetween pixels at a block boundary and an inner block side are differentfrom each other. And, “QP” is a quantization parameter of a macro blockto which the pixel f(i,j) belongs.

Even though the restored video from which blocking is removed usingFormulas 1-8 and 1-9, the value of the regularization parameter requiresa floating-point operation so as to become a problem in aspect ofcalculation quantity of the filter to remove the “blocking”.

Therefore, Formulas 1-8 and 1-9 are transformed into the followingFormulas 1-10 and 1-11 for integer operation.

$\begin{matrix}{{f\left( {i,j} \right)} = \frac{{\left( {2^{9} - {\alpha_{P} \times 2^{8}} - {\alpha_{N} \times 2^{8}}} \right){g\left( {i,j} \right)}} + {\alpha_{P} \times 2^{8} \times {g\left( {i,{j - 1}} \right)}} + {\alpha_{N} \times 2^{8} \times {g\left( {i,{j + 1}} \right)}}}{\;}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}10} \right\rbrack \\{{\beta_{P} = {{\alpha_{P} \times 2^{8}} = \frac{2^{8} \times K_{P}{QP}^{2}}{{K_{P}{QP}^{2}} + \left\{ {{g\left( {i,j} \right)} - {g\left( {i,{j - 1}} \right\}}^{2}} \right.}}},{\beta_{N} = {{\alpha_{N} \times 2^{8}} = \frac{2^{8} \times K_{N}{QP}^{2}}{{K_{N}{QP}^{2}} + \left\{ {{g\left( {i,j} \right)} - {g\left( {i,{j - 1}} \right\}}^{2}} \right.}}}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}11} \right\rbrack\end{matrix}$

In this case, β_(P) and β_(N), as defined in Formula 1-11, are variablesattained by multiplying α_(P) and α_(N) by 2⁸, respectively. Namely,β_(P) and β_(N) are parameters for making α_(P) and α_(N) into integers.

In an algorithm according to the present invention, the regularizationparameters as integer forms using Formula 1-11 are generated, and thenstored as a look-up table form to be used.

Hence, Formula 1-10 for blocking filtering can be found by knowing thepixel f(i,j) and locations and quantization parameter values of twopixels adjacent to the pixel f(i,j).

Meanwhile, in the loop filtering process of the present invention, avideo of which blocking in a direction of a horizontal axis is removedby loop-filtering a block of a moving picture is attained, and then thevideo is loop-filtered in a direction of a vertical axis so as toprovide the video of which blocking in a direction of a vertical axis isremoved.

FIG. 1 illustrates a flowchart of a loop filtering process according toan embodiment of the present invention.

Referring to FIG. 1, a loop filtering method according to an embodimentof the present invention includes the steps of determining a strength ofa block to be processed in accordance with a coded pattern and aquantization step size of each 4×4 block when a block of a movingpicture is loop-filtered in directions of its horizontal and verticalaxes (S11), carrying out a loop filtering in accordance with thedetermined strength of the corresponding block (S12), and carrying outan additional loop filtering at a boundary region of a macro block sincea relatively big blocking exists at the boundary region of the macroblock (S13). This is explained in detail by referring to FIG. 2A, FIG.2B, and FIG. 3.

FIGS. 2A and 2B illustrate block diagrams indicating block locations inhorizontal and vertical directions according to an embodiment of thepresent invention, respectively, and FIG. 3 illustrates a diagram ofpixel locations of adjacent blocks according to an embodiment of thepresent invention, in which block locations in directions of horizontaland vertical axes are shown.

First of all, the step of determining the block strength St and afiltering state is explained as follows.

The step of determining the block strength St and a filtering state is aprocess for determining the H.26L system as standards, for which TML(test model long-term number 5) is in progress. In this case, the TMLcompresses a video by taking a 4×4 block as a reference.

The TML (test model long-term) moving picture compression system uses atechnique of removing spatial and temporal redundancy information froman encoder so as to transmit spatially & temporally compressedinformation and additional information required for decoding. Of course,in order to remove the redundancy information on a temporal domain, MPEG(moving picture expert group) and H.263 motion compensation systemaccording to the related art are used. The technique of removing theblocking using a basic mechanism of such a TML system is generallytreated by 4×4 block unit.

In this case, H.263 and H.26L are moving picture compression standardsproposed by ITU-T (international telecommunicationsunion-telecommunication).

The system for removing the blocking according to an embodiment of thepresent invention may follow the system of the related art. Yet, thefiltering state is determined by the following pre-treatment steps 1 to3 since the degree of the blocking differs in accordance with a codedblock pattern, a quantization step size, and a motion vector size.

[Step 1]

Strength St of each 4×4 block is initialized by ‘0’(St=0).

[Step 2]

If each of the 4×4 blocks is an intra-coded block or has a non-zerotransform coefficient, St=max(St,2).

[Step 3]

If a motion vector difference between respective motion vectors for thetwo blocks block1 and block2 in FIG. 2( a) and FIG. 2( b) is equal to orgreater than ‘1’, St=Max(St,1).

In this case, max(a,b) means a maximum value of ‘a’ and ‘b’. Namely,max(a,b) is a function for selecting a greater value from ‘a’ and ‘b’.

Moreover, after the strength St of each of the blocks has beendetermined, the loop filtering is carried out in accordance with thequantization size of each of the blocks.

For carrying out the loop filtering, the quantization sizes of the twoblocks block1 and block2 are as follows.

First case is that the quantization value of the block block1 is smallerthan ‘21’ and the strengths of the two blocks block1 and block2 are not‘0’. And, second case is that the quantization value of the block block1is greater than ‘20’ and the strength St of one of the two blocks block1and block2 is not ‘0’.

In this case, the filtering state is determined differently inaccordance with the quantization size because an adaptive treatment isperformed due to the dominating generation of blocking in case of agreat quantization size.

Therefore, when the conditions for the pre-treatment steps 1 to 3 aremet, the loop filtering process is carried out by the following manner.

First, pixel locations of two blocks block1 and block2 are shown in FIG.3. If the above-defined filtering state of each of the blocks issatisfied, values of pixels c, d, e, and f are found by the followingFormula 2-1 (for calculating filter coefficient values to be used inFormula 2-5).r1=reg{St ₁ ,QP,abs(b−c)},r2=reg{St ₁ ,QP,abs(c−d)},r3=reg{max(St ₁+1,St ₂+1),QP,abs(d−e)},r4=reg{St ₂ ,QP,abs(e−f)},r5=reg{St ₂ ,QP,abs(f−g)}  [Formula 2-1]

From Formula 2-1, reg(.) is defined by the following Formula 2-2.

$\begin{matrix}{{{{reg}\left( {{St},{QP},{Diff}} \right)} = \frac{256 \times {M({St})} \times {{DPQ}({QP})}}{{{Diff} \times {Diff}} + {{M({St})} \times {{DQP}({QP})}}}},} & \left\lbrack {{Formula}\mspace{14mu} 2\text{-}2} \right\rbrack\end{matrix}$where Diff is a random value and ‘abs’ indicates a function of absolutevalue.

And, M(St) is defined by the following Formula 2-3.M(St=0)=⅙,M(St=1)= 4/16,M(St=2)= 8/16,M(St=3)=1  {Formula 2-3}

And, DQP(QP) is explained by referring to FIG. 4 as follows.

FIG. 4 illustrates a table of LQP values for determining DQP(QP) valuesaccording to an embodiment of the present invention. Namely, DQP(QP) isdefined by the following Formula 2-4, and can be found by substitutingthe LQP values shown in the table of FIG. 4. LQP is a quantization tabledefined in H.26L, and DQP defines a quantization error range inaccordance with the quantization table defined in H.26L as defined byFormula 2-4. Namely, the LQP values are “620”, “553”, “429”, . . . ,“24”, and “22” if the quantization variables QP are “0”, “1”, “2”, . . ., “30”, and “31”, respectively. Namely, as the quantization variable (0,1, 2, . . . , 30, 31) increases sequentially, the LQP value decreaseswith a predetermined value (620, 553, 492, . . . , 24, and 22).

$\begin{matrix}{{{DQP}({QP})} = \frac{676}{{LQP}({QP})}} & \left\lbrack {{Formula}\mspace{14mu} 2\text{-}4} \right\rbrack\end{matrix}$

Formula 1-11 is equivalent to Formula 2-2, and QP², K_(P) (or K_(N)),and {g(i,j)−g(i,j−1)} in Formula 1-11 are parameters equivalent toDQP(QP), M(St), and Diff in Formula 2-2, respectively.

Hence, the filtered values of the pixels c, d, e, and f are calculatedby the following Formula 2-5 using the defined parameters.c _(mod)={(512−r ₁ −r ₂)×c+r ₁ ×b+r ₂ ×d+256}>>9d _(mod)={(512−r ₂ −r ₃)×d+r ₂ ×c+r ₃ ×e+256}>>9e _(mod)={(512−r ₃ −r ₄)×c+r ₃ ×e+r ₄ ×f+256}>>9f _(mod)={(512−r ₄ −r ₅)×f+r ₄ ×e+r ₅ ×g+256}>>9,  [Formula 2-5]where “>>” means a bit shift, and C_(mod), d_(mod), e_(mod), and f_(mod)mean values of filtering c˜f having the blocking shown in FIG. 3.

Meanwhile, the result from the above-explained processing brings aboutan effect of removing the blocking and ringing with satisfaction. Yet,micro blocking still exists in the boundary region of the macro blockconstituted by 16×16 block unit. In order to settle the micro blocking,an additional filtering, as shown in Formula 2-6, is carried out on thepixel lying at the boundary region of the macro block.d _(mod)=(c _(mod)+13×d _(mod)+2×e _(mod)+8)>>4e _(mod)=(2×d _(mod)+13×e _(mod) +f _(mod)+8)>>4,if QP>16 and d and e are macro block boundary pixels.

Accordingly, the present invention carries out a real-time processing ofremoving a blocking and a ringing phenomenon in accordance with a pixellocation, a coded block pattern, a quantization step size, and a motionvector size in a digital video appliance using the H.26L moving picturecompression technique, thereby enabling to improve a video quality of acompression image demanding a low bit rate or high-speed processing.

The foregoing embodiments are merely exemplary and are not to beconstrued as limiting the present invention. The present teachings canbe readily applied to other types of apparatuses. The description of thepresent invention is intended to be illustrative, and not to limit thescope of the claims. Many alternatives, modifications, and variationswill be apparent to those skilled in the art.

1. A filtering method of a block boundary region in a loop filteringsystem, the method comprising: determining a filtering strength withrespect to the block boundary region; and filtering the block boundaryregion based on the determined filtering strength, wherein, in thefiltering strength determining step, the filtering strength isdetermined as a first strength based on whether blocks corresponding toa block boundary include non-zero transform coefficient information, orthe filtering strength is determined as a second strength based onmotion vector information of blocks corresponding to of the blockboundary region.
 2. The method of claim 1, wherein the filteringstrength is determined as the second strength based on a differencebetween motion vectors of the blocks corresponding to of the blockboundary region.